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Objectives: The drug utilization review (DUR) system, which checks any conflict event of medications, contributes to im- 
prove patient safely. One of the important barriers in its adoption is doctors' resistance. This study aimed to analyze the im- 
pacts of doctors' resistance on the success of the DUR system. Methods: This study adopted an augmented the DeLone and 
McLean Information System (D&M IS) Success Model (2003), which used doctors' resistance as a socio-technological mea- 
sure. This study framework is the same as that of the D&M IS Success Model in that it is based on qualities, such as system, 
information, and services. The major difference is that this study excluded the variable 'use' because it was not statistically 
significant for mandatory systems. A survey of doctors who used computers to enter prescriptions was conducted at a Korean 
tertiary hospital in February 2012. Results: This study is very meaningful in that it is the first study to explore the success fac- 
tors of the DUR system associated with doctors' resistance. Doctors' resistance to the DUR system was not statistically associ- 
ated with user usefulness, whereas it affected user satisfaction. Conclusions: The results indicate that doctors still complain of 
discomfort in using the DUR system in the outpatient clinical setting, even though they admit that it contributes to patient 
safety. To mitigate doctors' resistance and raise user satisfaction, more opinions from doctors regarding the DUR system have 
to be considered and have to be reflected in the system. 
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I. Introduction 

Information technology (IT) in healthcare has become in- 
creasingly prevalent since the late 20th century [1,2]. Health- 
care information systems (HISs), a typical application of IT 
in healthcare, were adopted to support Medicare and Medic- 
aid in the United States in the late 1960s. However, their role 
in improving patient safety was not recognized until a report 
by the Institute of Medicine, "To err is human: building a 
safer health system", was published in 1999. According to the 
report, at least 44,000 people died due to medical errors in 
hospitals every year. Since then, HIS has been regarded as a 
tool to improve patient safely [3] . A drug utilization review 
(DUR) system is a knowledge-based practice of the HIS 
which gives time-pressed doctors invaluable information 
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when prescribing drugs, and it can prevent any harm to pa- 
tients from medication errors [4] . 

Unfortunately, the healthcare sector has been slower in 
adoption of HISs in comparison with other industries, such 
as banking and automotive industries [5]. There are a variety 
of substantial barriers. One of the major obstacles is doctors' 
resistance [5-10]. To successfully implement the HIS in the 
real healthcare setting, there is a strong need to study how 
such doctors' resistance affects satisfaction and usefulness 
after its implementation. 

There have been a few formal usability studies on experi- 
enced doctors as users. However, there have been few analy- 
sis cases on the impact of doctors' resistance with regard to 
success of the DUR system. In 1992, DeLone and McLean 
presented the DeLone and McLean (D&M) Information 
System (IS) Success Model which could be used to assess the 
success of an IT system based on multi-dimensional con- 
structs rather than by single factor [11]. Thus, the present 
study aimed to analyze the impacts of doctors' resistance on 
the success of DUR systems based on an augmented D&M 
IS Success Model (2003), which used doctors' resistance as a 
socio-technological measure. 

II. Methods 

1. Study Site: Samsung Medical Center 

Samsung Medical Center (SMC) is a Korean tertiary hos- 
pital with a capacity of 2000 beds. The Korean government 
has guided all hospitals to implement the concurrent DUR 
(cDUR) system by the end of 2011. To follow this govern- 
ment guideline, the SMC organized a task force (TF) in July 
2011. The TF included doctors, pharmacists, nurses, admin- 
istrators, and IT specialists. Its mission was to design, devel- 
op, and implement the cDUR system before 2012. The cDUR 
system has been tightly integrated with the SMC's computer- 
ized physician order entry (CPOE) system in the outpatient 
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clinical setting. Through gathering opinions from several 
TF meetings, the cDUR system was developed in December 

2011 and adopted for outpatients since January 1, 2012 with- 
out any trouble. According to the government guidelines, the 
checking of DUR conflict events is mandatory in hospitals in 
order for its benefits to be realized [12]. 

2. Study Setting: DUR System 

In Korea, the concept of the DUR was introduced in the 
1980s, but it had not been seriously discussed until early 
2000. At the end of 2011, the Korean government issued a 
guideline for the cDUR system to be adopted in all hospitals. 
If the guideline is not followed, the National Health Insur- 
ance Review Agency (NHIRA) may retrospectively limit 
reimbursement of the hospital's claim. This system automati- 
cally checks whether a prescription meets the criteria de- 
veloped by NHIRA at the time of prescribing. DUR conflict 
events are based on drug-drug interactions, such as dupli- 
cate medication, contraindicated drug interaction, etc., for 
outpatients. Implementation of the DUR system by Korea 
hospitals is the first case in the world in which all healthcare 
providers have used the system nationwide [13]. 

The first step of the cDUR system is that the doctor enters 
drug information into the CPOE system (®), as shown 
in Figure 1. Next, the entered data are exchanged with the 
national DUR server to check for conflict events (©). The 
checked results are returned to the doctor's computer display 
((D). When there are no conflict events, the prescription is 
stored into the national DUR database, and the prescription 
process is finished (©). In the case of DUR conflict events, 
the doctor has to enter the reason why the outpatient needs 
the drugs (©). When the doctor does not have any reason 
to proceed with the prescription, the prescription has to be 
canceled or changed (©). 

Real prescription data during one week from January 2 to 8, 

2012 was analyzed immediately after adoption of the cDUR 
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Figure 1. Architecture of the concur- 
rent drug utilization review 
(cDUR) system in Korea. 
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system. During the period, the number of outpatients was 
40,908. Among them, 21,701 (53.0%) were prescribed drugs 
by doctors. The number of DUR conflict events was 1,752, 
which comprised 4.3% of outpatients. Duplicate medication 
was found in 99.3% of DUR conflict events, and contraindi- 
cated drug interaction accounted for 0.7% [11]. The time re- 
quired for entering a reason for the DUR conflict was about 
31.6 seconds in the outpatient clinical setting. 

3. Research Model and Hypotheses 

According to a literature review, there have been several 
studies on the D&M IS Success Model of the HIS [14-18]. 
Studies regarding the success of the DUR system were rare, 
and there was also a lack of analysis of the impact of doctors' 
resistance. As seen in the system architecture (Figure 1), the 
DUR system is very complex, and this leads to an increase 
in the time required for the prescription process in the clini- 
cal setting. This complicated work should be observed to 
be helpful to improve patient safety. However, doctors may 
consider it as a laborious task in their already time-pressing 
outpatient clinical setting, although they recognize its useful- 
ness. Thus, the process may invoke doctors' resistance even 
if they use it under compulsion. Thus, this study focused 
on the success of the DUR system based on the augmented 
D&M IS Success Model, which used doctors' resistance as 
a socio-technological measure. This framework is the same 
as that of the D&M IS Success Model in that it is based on 
qualities, such as system, information, and services [19]. The 
major difference is that this study excluded the variable 'use' 
because it was not statistically significant for mandatory sys- 
tems [20] . Another very specialized construct is newly added 
doctors' resistance in this study to examine whether this has 
an impact on user satisfaction or user usefulness in a statisti- 
cally significant fashion or not. The net benefit was specified 
as 'user usefulness' to reflect doctors' viewpoint, since doc- 
tors are the main users of the DUR system. Pertaining to net 
benefit, benefits for patients or organizations may also be 
considered. Any improvement of patient service, satisfaction 
of patients, cost issues of organizations, etc., can be examples 
of net benefit. However, this study selected 'user useful- 
ness' in that it is meaningful for the system to be evaluated 
in stakeholders' interests, as recommended by Seddon et al. 
[21]. Hence, this study tried to measure whether the DUR 
system could be useful in improving patient safety or quality 
for doctors to care patients or not. Review of previous stud- 
ies led to the conclusion that DUR systems could be consid- 
ered in relation to the following possible benefits: 'useful for 
making prescription, 'improving decision-making', and 'giv- 
ing doctors positive impact'. Considering these aspects, the 
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Figure 2. Research model and hypotheses. DUR: drug utilization 
review. 



research model of this study is presented in Figure 2. 

Hypothesis 1 (HI): The overall quality of the DUR system 
positively impacts user satisfaction but user satisfaction is 
negatively impacted by doctors' resistance. 
Hl-1: System quality positively impacts user satisfaction. 
HI -2: Information quality positively impacts user satisfaction. 
Hl-3: Service quality positively impacts user satisfaction. 
Hl-4: Doctors' resistance negatively impacts user satisfaction. 

Hypothesis 2 (H2): User satisfaction positively impacts 
user usefulness but user usefulness is negatively impacted 
by doctors' resistance. 

H2-1: User satisfaction positively impacts user usefulness. 
H2-2: Doctors' resistance negatively impacts user usefulness. 

Hypothesis 3 (H3): Both user satisfaction and user useful- 
ness negatively impact doctors' resistance. 
H3-1: User satisfaction negatively impacts doctors' resistance. 
H3-2: User usefulness negatively impacts doctors' resistance. 

4. Variables and Measures 

The questionnaire used in this study was composed of six 
variables based on the hypotheses (Appendix 1). In the sys- 
tem quality variable, the measured items were ease of use, 
response time, and system reliability. In the information 
quality variable, the measured items were understandability, 
accuracy, and timeliness. In the service quality variable, the 
measured items were sincerity, right time, and high service 
quality. The measured items in the user satisfaction variable 
were overall satisfaction, user friendliness, and positive at- 
titude. In the case of doctors' resistance, the measured items 
included workload increase, time consumption and no 
tangible benefits. Usefulness, decision-making, and positive 
benefits were included in the variable measuring user useful- 
ness. 

The measures for system quality, information quality, user 
satisfaction, and user usefulness were mainly based on stud- 
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ies by van der Meijden et al. [22] in 2003, Jen and Chao [14] 
in 2008, and Petter and Fruhling [16] in 2011. The measures 
for doctors' resistance were based on challenges or bar- 
riers described by Lee et al. [8] in 1996, Sittig et al. [5] in 
1999, Murff and Kannry [23] in 2001, Carroll et al. [9] in 
2002, Bates and Gawande [7] in 2003, Berger and Kichak 
[6] in 2004, Aarts et al. [10] in 2007, and Ventura et al. [24] 
in 2011. A 5-point Likert-type scale was used for all items 
with 1 being 'strongly disagree', 3 being 'neutral', and 5 being 
'strongly agree'. 

5. Survey and Data Analysis 

A survey of doctors who have used computers to enter pre- 
scriptions was conducted over a period of 3 weeks from 
February 22 to March 13, 2012. The questionnaire was based 
on the self-reported usage pattern. An informal informed 
consent was included at the head of the questionnaire. IBM 
SPSS Statistics ver. 20 (IBM, Armonk, NY, USA) was used 
for data analysis. Although the variables and measures 
were already validated in previous researches, validity was 
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checked through factor analysis as a confirmatory method 
because some of them were newly combined in this study. 
To check the validity between variables and measures, factor 
analysis with IBM SPSS Statistics ver. 20 was performed. The 
analysis of results was based on convergent validity, discrim- 
inant validity, and nomological validity [25] . The extraction 
method of factor analysis was principle component analysis, 
and the rotation method was Equamax with Kaiser normal- 
ization. After analysis, measures with low construct validity, 
low convergent validity, or low discriminant validity were 
excluded. With Cronbach's a, the reliability of the measures 
was assessed. Multiple linear regression analysis was used 
to examine the hypotheses. In addition, in-depth interviews 
with doctors, who have rich experience in leading IT proj- 
ects at the hospital, were conducted to discuss the results of 
data analysis. 

The Institutional Review Board (IRB) review has been 
obligatory for all research related to human subjects since 
February 1, 2013 in Korea. According to the IRB office, this 
study is eligible for a waiver of IRB review because this study 



Table 1. Results of factor analysis 



Variable 


Measure 


1 . System 


2. Information 


3. Service 


4. User 


5. Doctors' 


6. User 




quality 


quality 


quality 


satisfaction 


resistance 


usefulness 


Al, ease to use 


0.318 


0.239 


0.184 


0.799 


0.170 


0.174 


A2, response time 


0.807 


-0.042 


0.234 


0.168 


0.070 


0.300 


A3, reliability 


0.861 


0.032 


-0.029 


0.017 


0.077 


-0.329 


Bl, understandability 


-0.101 


0.710 


0.260 


0.437 


-0.041 


-0.310 


B2, accuracy 


0.012 


0.820 


0.201 


-0.118 


0.303 


0.204 


B3, timeliness 


0.431 


0.559 


0.518 


0.064 


0.112 


0.201 


CI, sincerity 


0.110 


0.307 


0.839 


0.310 


0.057 


0.001 


C2, right time 


0.252 


0.305 


0.817 


0.159 


0.127 


0.230 


C3, high service quality 


0.167 


0.284 


0.782 


0.335 


0.132 


0.145 


Dl, overall satisfaction 


0.099 


0.136 


0.404 


0.732 


0.281 


0.208 


D2, user friendliness 


0.130 


0.101 


0.374 


0.685 


0.299 


0.283 


D3, positive attitude 


0.114 


0.387 


0.260 


0.492 


0.248 


0.546 


El, work load increase 


0.057 


0.259 


-0.084 


0.161 


0.881 


0.189 


E2, time consumption 


0.119 


0.029 


0.170 


0.116 


0.933 


-0.039 


E3, no tangible benefits 


0.179 


-0.086 


0.196 


0.556 


0.293 


0.519 


Fl, usefulness 


0.166 


0.367 


0.156 


0.380 


0.358 


0.585 


F2, decision-making 


-0.027 


0.024 


0.055 


0.149 


0.177 


0.845 


F3, positive benefits 


0.050 


0.162 


0.198 


0.194 


0.032 


0.835 



The method for factor extraction is Principal Component Analysis, and the method for rotation is Equamax with Kaiser Normalization. Rotation 
converged in 14 iterations. 

The digits in bold style mean that the measures are relevant to the intended variables based on the literature reviews. However, the measures in 
italic style, such as Al ease to use, D3 positive attitude, and E3 no tangible benefits, are excluded because of their irrelevances. 
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did not store any personal information which could be used 
to identify respondents. 

III. Results 

1. Validity and Reliability 

Of 650 doctors who entered prescriptions in January 2012, 
54 answered the questionnaires. Of the returned 54 ques- 
tionnaires, 20 were invalidated because 7 were returned 
incomplete, and 13 were insincere. This yielded a total of 34 
usable questionnaires for data analysis with a validation rate 
of 63%. The response rate was 5.3%, that is, 34 valid surveys 
out of 650 doctors. Respondents of valid questionnaires were 
68% male; 71% were between the ages of 30 and 50; 50% 
were professors, 15% were fellows, and 35% were residents; 
47% had working experience of more than 10 years, 12% had 
5 to 10 years, and 41% had less than 5 years. The distribution 
of specialties was 32% in internal medicine, 35% in surgery, 
9% in pediatrics, 6% in neurology, 6% in family medicine, 
and 12% in other areas. Even though the response rate was 
very low, this distribution was very similar to the distribu- 
tion of doctors in the hospital. Therefore, this survey was 
able to cover a representative sample of the hospital. 

According to the results of checking the validity between 
variables and measures, as shown in Table 1, response time 
and reliability are relevant to system quality; understandabil- 
ity, accuracy, and timeliness are relevant to information qual- 
ity; sincerity, right time, and high service quality are relevant 
to service quality; overall satisfaction and user friendliness 
are relevant to user satisfaction; increase in workload and 
time consumption are relevant to doctors' resistance; useful- 
ness, decision-making, and positive benefits are relevant to 
user usefulness. 

However, ease of use was relevant to user satisfaction rather 
than system quality in this study, although it was the most 
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frequently addressed measure of system quality. Thus, it was 
excluded from further analysis. Positive attitude was not rel- 
evant to user satisfaction, even though it was recommended 
as a measure of user satisfaction [22]. It was also excluded 
from this analysis. From the perspective of nomological 
validity, no tangible benefits were also excluded since it is 
strongly related to doctors' resistance rather than to user sat- 
isfaction according to the literature review [6]. 

According to a result of the reliability assessment with 6 
variables and 15 measures except ease of use, positive at- 
titude and no tangible benefits, Cronbach's a for the 5 vari- 
ables indicated good reliability with reliability values higher 
than 0.7 representing the minimal standard [25]. System 
quality had a slightly weak reliability of 0.640. The mean, 
standard deviation, and Cronbach's a for the variables are 
listed in Table 2. The means of all variables were above 3.0. 
Pearson correlation coefficients indicated low and high exis- 
tent correlations among the variables examined for the DUR 
system. Among the variables, doctors' resistance showed a 
strong correlation with user satisfaction, whereas it was less 
correlated with user usefulness. 

2. Hypothesis Test 

Typically, studies on IS success use structural equation mod- 
eling (SEM) for large sample sizes. However, due to the small 
sample size of the present study, multiple linear regression, 
rather than SEM, was adopted to examine the relationships 
among the hypotheses in this research. The results of the re- 
gression analysis are presented in Table 3. 

System quality, information quality, service quality, and 
doctors' resistance (HI) did not behave as hypothesized. For 
user satisfaction, the regression model was significant (R 2 = 
0.529, p = 0.000). Of the 4 variables, only service quality was 
significant with a positive beta ((3 = 0.61 l,p = 0.001), where- 
as doctors' resistance was negatively significant ((3 = -0.279, 



Table 2. Means and correlation of variables 



Variable 


No. of 
measures 


Mean 


SD 


Cronbach's a - 


1 


2 


Correlation 
3 4 


5 6 


1. System quality 


2 


3.338 


0.776 


0.640 


1.000 








2. Information quality 


3 


3.373 


0.596 


0.742 


0.200 


1.000 






3. Service quality 


3 


3.392 


0.835 


0.938 


0.327 


0.672** 


1.000 




4. User satisfaction 


2 


3.074 


0.799 


0.938 


0.276 


0.450** 


0.667** 1.000 




5. Doctors' resistance 


2 


3.618 


0.871 


0.881 


-0.195 


-0.350* 


-0.267 -0.448** 


1.000 


6. User usefulness 


3 


3.284 


0.735 


0.829 


0.136 


0.374* 


0.461** 0.600** 


-0.385* 1.000 



SD: standard deviation. 
*p < 0.05, **p < 0.01. 



Vol.20 • No. 2 • April 2014 



www.e-hir.org 103 



Jong Soo Choi et a I Healthcare Informatics Research pHR 



Table 3. The results of regression analysis 







Hypotheses 


R 


P 




Results 


HI 


Hl-1 


System quality 


-> User satisfaction 0.529** 


0.027 


Not supported 


Partially supported 




Hl-2 


Information quality 




-0.122 


Not supported 






Hl-3 


Service quality 




0.611** 


Supported 






Hl-4 


Doctors' resistance 




-0.279* 


Supported 




H2 


H2-1 


User satisfaction 


-> User usefulness 0.377** 


0.492** 


Supported 


Partially supported 




H2-2 


Doctors' resistance 




-0.123 


Not supported 




H3 


H3-1 


User satisfaction 


-> Doctors' resistance 0.222* 


-0.370*** 


Not supported 


Not supported 




H3-2 


User usefulness 




-0.216 


Not supported 





*p < 0.05, **p < 0.01, ***p < 0.1. 



p = 0.035). Thus, hypothesis HI was partially supported, as 
shown in Table 3. 

Similar to hypothesis HI, hypothesis H2 was partially sup- 
ported, as shown in Table 3. With regard to user usefulness, 
the regression model was significant (R 2 = 0.377, p = 0.001). 
User satisfaction had a significant positive association ((} = 
0.429, p = 0.002), while doctors' resistance had no significant 
association. 

Hypothesis H3 was not supported, as shown in Table 3. For 
doctors' resistance, the regression model was not strongly 
significant (R 2 = 0.222, p = 0.020) and neither user satisfac- 
tion nor user usefulness was associated with doctors' resis- 
tance, while user satisfaction was suggestively significant ((3 
= -0.370, p = 0.097). 

IV. Discussion 

This study aimed to analyze the impact of doctors' resistance 
on the DUR system, which is a system that checks any con- 
flict events of medications when doctors prescribe drugs in 
the outpatient clinical setting. An augmented D&M IS Suc- 
cess Model with doctors' resistance was employed to evalu- 
ate the DUR system. As expected, the D&M Success Model 
was a useful multi-dimensional tool to evaluate a specialized 
HIS, such as the DUR system, and these empirical results 
provide considerable support for the model. As mentioned 
before, this study is very meaningful in that it is the first 
study to explore the success factors of the DUR system asso- 
ciated with doctors' resistance. 

The time required to enter a reason for the DUR conflict 
was about 31.6 seconds in the outpatient clinical setting. In 
Korea, doctors work in time-pressed environments such that 
only 3 to 5 minutes are allowed for one outpatient examina- 
tion. Due to the DUR conflict check, an additional half min- 
ute was required to examine one outpatient. Thus, doctors 




User 
satisfaction 



H2-1 



i H3-1 



P = -0.370 
p = 0.097 




User 
usefulness 



Doctors' 
resistance 



Figure 3. The results of hypotheses test. DUR: drug utilization 
review. 

were concerned that the DUR system increased their ad- 
ditional workload when prescribing drugs, and it disrupted 
their sense of professionalism. This attitude could explain 
why doctors have anxiety regarding the introduction of new 
IT system and why the healthcare industry is late to adopt 
new technology in comparison with other industries, as 
described in previous studies [5,8-10,23]. In this case, they 
express dissatisfaction with the DUR system, even though it 
is useful for patient safety. This is the reason why the doctors 
relentlessly opposed the adoption of the DUR system by Ko- 
rea government. 

The results show that: first, service quality is positively associ- 
ated with user satisfaction; second, doctors' resistance is nega- 
tively associated with user satisfaction, whereas it is not asso- 
ciated with user usefulness; third, user satisfaction is positively 
associated with user usefulness; finally, neither user satisfac- 
tion nor user usefulness are statistically significant in associa- 
tion with doctors' resistance (Figure 3). In-depth interviews 
provide an effective insight into what things have happened 
and would happen [16]. We conducted in-depth interviews 
with several doctors to provide relevant explanations on re- 
sults of hypothesis test in the research finding section. Doctors 
who participated in the interviews had extensive experience 
in leading IT projects at the hospital, especially computer- 
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supported care services and theoretical insights from medical 
sociology. They included 3 doctors from internal medicine, 2 
from surgery, and 1 from pediatrics. 

Only service quality among the overall quality of the DUR 
system, as measured by system quality, information quality, 
and service quality, was significantly associated with user sat- 
isfaction in this study (Hl-3). Considering that both system 
quality and information quality are strong predictors for IS 
success [16], the current study results are inconsistent with 
those of most previous studies. In the healthcare domain, us- 
ers highly value systemic response time, reliability, precision, 
and timeliness in their work [14] . The in-depth interview re- 
sults revealed that HIS has to basically ensure those qualities, 
of course. However, those interviewed expressed that what is 
more important is how to make rapid response to the prob- 
lem in the time-critical outpatient setting. It appeared that 
respondents in this study focused more on aspects of service 
quality, such as service sincerity, right time service, and high 
service quality. They also said that they could not overlook 
the importance of system and information quality, although 
these did not have an influence on user satisfaction in this 
study. 

We found that doctors' resistance is negatively associated 
with user satisfaction (Hl-4). This may be explained by 
the fact that the DUR system increased doctors' workloads 
because they had to enter specific reasons for every DUR 
conflict event popup. In case of a DUR conflict event, it 
took doctors an additional 0.5 minutes to enter the reason. 
Considering that only 3 to 5 minutes are used to examine 
one outpatient in Korea, this additional time cannot be con- 
sidered short. Doctors, who participated in the interview, 
also expressed annoyance with DUR conflict alarms and 
felt that entering reasons was tiresome. This opinion is very 
similar to descriptions from other previous studies [5,8,9,23] . 
According to previous studies, the HIS had increased con- 
sultation times due to causes, such as DUR conflict event 
popup. Doctors expressed that they disrupted their clinical 
workflow and decreased their professionalism. Sometimes, 
this led to a boycott of HIS adoption as a form of resistance. 
Thus, it is quite natural that doctors' resistance is negatively 
associated with user satisfaction (Hl-4). Further research is 
needed to identify whether the resistance comes simply from 
inconvenience or other reasons, such as invasion of their 
professionalism. 

One of the most commonly measured attributes in the 
D&M Model is user satisfaction [17]. We found that user 
satisfaction positively affected user usefulness (H2-1). The 
result is consistent with most previous HIS studies in that 
user satisfaction might be a strong predictor of individual 
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impact, such as user usefulness [14-16]. An interesting find- 
ing in this study is that doctors' resistance is not associated 
with user usefulness (H2-2). With regard to the impact on 
the individual of the D&M IS Success Model, the stakehold- 
ers to be evaluated were doctors who use the DUR system 
and user usefulness which they felt was set as the impact in 
this particular context based on the research of Seddon et 
al. [21]. Usefulness was specified with the three measures of 
usefulness, decision-making, and positive benefits. User use- 
fulness is an objective fact, whereas satisfaction is the user's 
subjective feeling. Among those interviewed, some stated 
that doctors did not deny the usefulness of the DUR system, 
even though they suffered from its inconvenience. They also 
expressed that doctors might tolerate such inconvenience 
since the DUR system could contribute to improving the 
quality of care. This may be explained by the fact that doc- 
tors, who have strong professional ethics, put patient safety 
as their top priority. All those interviewed mentioned that it 
is necessary to improve convenience from the view point of 
doctors to lower barriers to HIS adoption in the healthcare 
industry [9]. 

However, those interviewed had a general impression that 
doctors' resistance may influence user usefulness if more 
valid questionnaires would be secured. Furthermore, they 
stated that doctors' resistance could affect the variable 'use', 
which was excluded from this study, if the DUR system was 
voluntary rather than mandatory according to the govern- 
ment guidelines. Under such conditions, doctors' resistance 
could negatively influence user usefulness as impacts user 
satisfaction. 

The results for H3 showed that neither user satisfaction nor 
user usefulness affected doctors' resistance. However, when 
we closely looked at the association between user satisfac- 
tion and doctors' resistance, it was suggestively significant 
because the p-value was 0.097, smaller than 0.1. Thus, it is 
difficult to say that user satisfaction was not at all associated 
with doctors' resistance. 

Although this study suggests some interesting findings 
that may be helpful in understanding the success factors of 
the DUR system, there are limitations to consider. First, the 
number of respondents for the analysis was relatively small. 
However, there is no reason to believe that there is systemat- 
ic response bias because the distribution of the respondents 
represents the population of users, although valid question- 
naires were received from only 5.3% of the entire population 
of users. Furthermore, the normality of the data collected 
was also retained because all Shapiro-Wilk statistics of the 3 
hypotheses were higher than 0.05 (HI = 0.980, H2 = 0.980, 
H3 = 0.955). In addition, only the DUR system was evalu- 
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ated in this study. Because this DUR system is a very specific 
clinical information system, more study cases are needed to 
gain more sufficient insight on the success of the HIS. An- 
other limitation is that the survey undertaken in this study 
was conducted in only a single organization. Although doc- 
tors of different hospitals use the same DUR system, they 
may have different views according to their clinical context. 
To establish a common model of IS success in healthcare, 
various opinions should be considered from different orga- 
nizations. Another limitation is that some measures are not 
relevant to constructs in substance. In case of 'ease of use', 
it is usually relevant to system quality [22]. In this study, 
doctors answered it for user satisfaction rather than system 
quality. 'Positive attitude' was not answered as a measure of 
user satisfaction and 'no tangible benefits' were not related 
to doctors' resistance. If they were answered as in a previous 
study [22], the results of multiple linear regression could be 
different from the results obtained in this study. 

In conclusion, it is evident that the HIS contributes to im- 
provement in the quality of patient care [2] . In particular, the 
DUR system is a very effective tool in preventing medical 
mistakes because it has a significant impact on doctors' pre- 
scribing habits. However, efforts to adopt such beneficial sys- 
tems are impeded by the resistance of doctors based on con- 
cerns about the negative effects of HIS. In this study based 
on the D&M IS Success Model, doctors' resistance to using 
the DUR system was not statistically significantly associated 
with user usefulness, whereas it affected user satisfaction. 
This means that doctors still complain of discomfort when 
using the DUR system in the outpatient clinical setting, even 
though they admit that it contributes to patient safety. All 
those interviewed suggest that, to mitigate doctors' resis- 
tance and improve user satisfaction, it is important to reflect 
opinions from doctors with strong professional ethics. 
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Appendix 1. Questionnaire 
[ Informed Consent ] 

This study is to analyze multi-dimensional DUR system success. This study does not store any personal information which 
can identify respondents. The collected data are only used for statistics analysis. Only if you agree to this study, please answer 
this survey after sign at the end of this line. ( ) 

[ General Information ] 

1) Date: 2012. . . 

2) Name: (If you DON'T want to, you do not have to write your name.) 

3) Department: 

4) Position: □ Professor, □ Fellow, □ Resident 

5) Sex: □ Male, □ Female 

6) Age: □ under 30, □ 30-39, □ 40-49, □ 50 and over 

7) Working Experience: □ under 5 years, □ 5-10 years, □ over 10 years 



Variables 


Items 


Measure 


DUR system quality System quality 


Al 


The DUR system is easy to use when prescribing, (ease of use) 




A2 


The response time of the DUR system is speedy, (response time) 




A3 


The DUR system is stable with no downtime, (reliability) 


Information quality 


Bl 


The information of the DUR system is easy to understand, (understandability) 




B2 


The information of the DUR system is accurate, (accuracy) 




B3 


The information of the DUR system is timely, (timeliness) 


Service quality 


CI 


The technical support service is sincere on users' need, (sincerity) 




C2 


The technical support service is prompt to users, (right time) 




C3 


The technical support service is rated highly, (high service quality) 


User satisfaction 


Dl 


Overall, the DUR system provides satisfaction, (overall satisfaction) 




D2 


The DUR system is user-friendly, (user friendliness) 




D3 


I keep a positive attitude toward the DUR system, (positive attitude) 


Doctors' resistance 


El 


The DUR system increases my workload, (workload increase) 




E2 


The DUR system leads to time consumption, (time consumption) 




E3 


It is hard to realize potential benefits such as patient safety, cost saving by us- 






ing the DUR system, (no tangible benefits) 


User usefulness 


Fl 


The DUR system is useful for making prescriptions, (usefulness) 




F2 


The DUR system improves decision-making when doctors make prescriptions. 






(decision-making) 




F3 


In general, using the DUR system has a positive impact for doctors, (positive 



benefits) 
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